Papers with fact checking
Fact Checking or Psycholinguistics: How to Distinguish Fake and True Claims? (D19-66)
Copied to clipboard
| Challenge: | Using psycholinguistic features to distinguish lies from true statements is a difficult task and a problem to be solved. |
| Approach: | They compare psycholinguistic text features with fact checking approaches to distinguish lies from true statements using data from a large ongoing study. |
| Outcome: | The proposed methods outperform both fact checking and human baselines but the accuracy is not high. |
Integrating Stance Detection and Fact Checking in a Unified Corpus (N18-2)
Copied to clipboard
| Challenge: | Existing methods for fact checking are not supported by existing datasets, which treat fact checking, document retrieval, source credibility, stance detection and rationale extraction as independent tasks. |
| Approach: | They propose to implement automatic fact checking on an Arabic fact checking corpus, which is the first of its kind. |
| Outcome: | The proposed approach is based on an Arabic fact checking corpus, the first of its kind. |
Interactive Evidence Detection: train state-of-the-art model out-of-domain or simple model interactively? (D19-66)
Copied to clipboard
| Challenge: | Evidence detection is a method that can be used to find evidence in a new topic without training data. |
| Approach: | They propose to use large amounts of out-of-domain data to train an evidence detection method . they simulate users who read source documents and label sentences they can use as evidence . |
| Outcome: | The proposed method outperforms state-of-the-art models on large out-of domain data. |
Tab-CQA: A Tabular Conversational Question Answering Dataset on Financial Reports (2023.acl-industry)
Copied to clipboard
| Challenge: | Existing conversational question answering datasets are usually constructed from unstructured texts in English. |
| Approach: | They propose a Chinese tabular conversational question answering dataset based on financial reports . they select 2,463 tables and manually generate 2,463, conversations with 35,494 QA pairs . |
| Outcome: | The proposed dataset is based on Chinese financial reports extracted from listed companies in the past 30 years. |
OpenRT: An Open-source Framework for Reasoning Over Tabular Data (2023.acl-demo)
Copied to clipboard
| Challenge: | Existing table pre-training methods are benchmarked on a limited number of datasets with varying configurations, resulting in a lack of unified, standardized, fair, and comprehensive comparison between methods. |
| Approach: | They propose to use OpenRT to reproduce existing table pre-training models and develop new models quickly. |
| Outcome: | The proposed framework reproduces existing table pre-training models and compares them against four question answering, one fact checking, and one faithful text generation datasets. |
Seeing Things from a Different Angle:Discovering Diverse Perspectives about Claims (N19-1)
Copied to clipboard
| Challenge: | a number of fact checking techniques are used to identify and eliminate biases in text data. |
| Approach: | They propose to use search engines to expand and diversify a dataset of claims, perspectives and evidence to address a selection bias. |
| Outcome: | The proposed approach outperforms existing methods in a language understanding task. |
A High Precision Pipeline for Financial Knowledge Graph Construction (2020.coling-main)
Copied to clipboard
Sarah Elhammadi, Laks V.S. Lakshmanan, Raymond Ng, Michael Simpson, Baoxing Huai, Zhefeng Wang, Lanjun Wang
| Challenge: | Knowledge graphs are a standard for structured knowledge representation in the Semantic Web. |
| Approach: | They propose to extract financial news articles into a knowledge graph by using a financial dictionary. |
| Outcome: | The proposed pipeline extracts 342,000 financial news articles with a precision of 78% at the top-100 extractions. |
Multi-Task Retrieval for Knowledge-Intensive Tasks (2021.acl-long)
Copied to clipboard
Jean Maillard, Vladimir Karpukhin, Fabio Petroni, Wen-tau Yih, Barlas Oguz, Veselin Stoyanov, Gargi Ghosh
| Challenge: | Knowledge-intensive tasks require large amounts of knowledge about the world . recent neural retrieval models achieve better results by learning directly from task-specific training data. |
| Approach: | They propose a multi-task trained neural retrieval model that can be universally trained on a wide variety of problems. |
| Outcome: | The proposed model outperforms specialised retrievers on a few-shot setting and matches or improves state-of-the-art on multiple benchmarks. |
Optimizing Retrieval-augmented Reader Models via Token Elimination (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for ODQA use a retrieval-augmented language model . a generative model can cause a significant bottleneck in decoding time . |
| Approach: | They propose to eliminate some of the retrieved information that might not contribute essential information to the answer generation process. |
| Outcome: | The proposed method reduces run-time by up to 62.2% with only 2% reduction in performance and improves performance. |
Towards Argument Mining for Social Good: A Survey (2021.acl-long)
Copied to clipboard
| Challenge: | Argument Mining is a social science-based approach to analysis and analysis of arguments. |
| Approach: | They propose a novel definition of argument quality which integrates the social science literature and the argument quality. |
| Outcome: | The proposed definition of argument quality integrates the social science literature and the argument quality debate. |
Factuality Assessment as Modal Dependency Parsing (2021.acl-long)
Copied to clipboard
| Challenge: | a critical step towards factuality assessment is to determine the factuality of events in text. |
| Approach: | They propose a modal dependency parsing task that assesses the factuality of events in text . they crowdsource a large-scale data set annotated with modal dependence structures . |
| Outcome: | The proposed model outperforms the pipeline model in factuality assessment . the proposed model is based on a crowdsourced dataset . |
TABi: Type-Aware Bi-Encoders for Open-Domain Entity Retrieval (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for entity retrieval use mention boundaries but unstructured text . state-of-the-art methods struggle to retrieve rare entities for ambiguous mentions . |
| Approach: | They propose a method to jointly train bi-encoders on knowledge graph types and unstructured text for entity retrieval for open-domain tasks. |
| Outcome: | The proposed method improves retrieval of rare entities on Ambiguous Entity Retrieval sets while maintaining strong overall retrieval performance on open-domain tasks. |
Re2G: Retrieve, Rerank, Generate (2022.naacl-main)
Copied to clipboard
Michael Glass, Gaetano Rossiello, Md Faisal Mahbub Chowdhury, Ankita Naik, Pengshan Cai, Alfio Gliozzo
| Challenge: | Recent models such as RAG and REALM incorporate retrieval into conditional generation. |
| Approach: | They propose a method that combines retrieval and reranking into a BART-based sequence-to-sequence generation. |
| Outcome: | The proposed model combines retrieval and reranking into a BART-based sequence-to-sequence generation. |
KILT: a Benchmark for Knowledge Intensive Language Tasks (2021.naacl-main)
Copied to clipboard
Fabio Petroni, Aleksandra Piktus, Angela Fan, Patrick Lewis, Majid Yazdani, Nicola De Cao, James Thorne, Yacine Jernite, Vladimir Karpukhin, Jean Maillard, Vassilis Plachouras, Tim Rocktäschel, Sebastian Riedel
| Challenge: | Existing models for knowledge-intensive language tasks require access to large, external knowledge sources. |
| Approach: | They propose a benchmark for knowledge-intensive language tasks (KILT) they test a shared dense vector index coupled with a seq2seq model to generate disambiguated text. |
| Outcome: | The proposed model outperforms tailor-made approaches on fact checking, open-domain question answering and dialog by generating disambiguated text. |
Search Augmented Instruction Learning (2023.findings-emnlp)
Copied to clipboard
Hongyin Luo, Tianhua Zhang, Yung-Sung Chuang, Yuan Gong, Yoon Kim, Xixin Wu, Helen Meng, James Glass
| Challenge: | Large language models (LLMs) have been significantly improved by instruction fine-tuning, but still lack transparency and the ability to utilize up-to-date knowledge and information. |
| Approach: | They propose a search-augmented instruction learning model which grounds the language generation and instruction following abilities on complex search results generated by in-house and external search engines. |
| Outcome: | The proposed model outperforms plain LLMs on zero-shot language tasks and can generate both natural and programming languages following natural language guidance and requests. |
Generating Label Cohesive and Well-Formed Adversarial Claims (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing work on adversarial triggers for fact checking models reveals weaknesses and flaws of models . universal adversarials often inadvertently invert the meaning of instances they are inserted in . |
| Approach: | They propose a method for automatically generating highly potent, well-formed, label cohesive claims for FC using universal adversarial triggers. |
| Outcome: | The proposed method maintains the directionality and semantic validity of the claim better than previous work on the FEVER dataset. |
Automated Fact Checking: Task Formulations, Methods and Future Directions (C18-1)
Copied to clipboard
| Challenge: | Recent research on fact checking has focused on misinformation . however, relevant papers and articles have been published in research communities that are unaware of each other and use inconsistent terminology. |
| Approach: | They propose avenues for future NLP research on automated fact checking . they highlight the use of evidence as an important distinguishing factor . |
| Outcome: | The proposed methods unify the task formulations and methodologies across papers and authors. |
Evaluating Entity Disambiguation and the Role of Popularity in Retrieval-Based NLP (2021.acl-long)
Copied to clipboard
| Challenge: | Existing studies show that retrievers underperform on rarer entities that share a name . open-domain tasks require a knowledge source to perform reasoning and produce an answer . |
| Approach: | They propose an evaluation benchmark for retrieving entities that share a name . they define Ambiguous Entity Retrieval sets as a collection of entities that have a common name - and query about those entities. |
| Outcome: | The proposed sets underperform on rarer entities that share a name . the retrievers exhibit popularity bias, and are twice as likely to retrieve erroneous documents . |
LogicalFactChecker: Leveraging Logical Operations for Fact Checking with Graph Module Network (2020.acl-main)
Copied to clipboard
Wanjun Zhong, Duyu Tang, Zhangyin Feng, Nan Duan, Ming Zhou, Ming Gong, Linjun Shou, Daxin Jiang, Jiahai Wang, Jian Yin
| Challenge: | Existing methods for fact checking textual statements are not yet available. |
| Approach: | They propose a neural network approach capable of leveraging logical operations for fact checking . they use a textual statement and semi-structured tables to generate a program from it . |
| Outcome: | The proposed approach achieves state-of-the-art performance on TABFACT dataset . it derives a program (a.k.a. logical form) of the statement in semantic parsing manner . |
Reasoning Over Semantic-Level Graph for Fact Checking (2020.acl-main)
Copied to clipboard
| Challenge: | Existing methods for fact checking use string concatenation or fusing features of isolated evidence sentences. |
| Approach: | They propose a method suitable for reasoning about the semantic-level structure of evidence . they use graph convolutional network and graph attention network to exploit the structure . |
| Outcome: | The proposed method improves claim verification accuracy and FEVER score on a benchmark dataset. |
How Entangled is Factuality and Deception in German? (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing research on deception detection and fact checking conflates factual accuracy with truthfulness . a belief-based deception framework defines texts as deceptive when there is a mismatch between what people say and what they truly believe . |
| Approach: | They assess if presumed patterns of deception generalize to German language texts . they gauge the impact of deceptiveness on the downstream task of fact checking . |
| Outcome: | The proposed framework disentangles deception when there is a mismatch between what people say and what they truly believe . the proposed framework does not find any correlation with established cues of deception . |
Generating Fact Checking Briefs (2020.emnlp-main)
Copied to clipboard
Angela Fan, Aleksandra Piktus, Fabio Petroni, Guillaume Wenzek, Marzieh Saeidi, Andreas Vlachos, Antoine Bordes, Sebastian Riedel
| Challenge: | Existing work has framed fact checking as classification, often supported by a claim as input. |
| Approach: | They propose to use natural language briefs to increase the accuracy of fact checking . they show that QABriefer increases the accuracy by 10% while QABries reduce time . |
| Outcome: | The proposed model increases the accuracy of crowdworkers by 10% while reducing the time required by 20%. |
Evaluating the Factual Consistency of Abstractive Text Summarization (2020.emnlp-main)
Copied to clipboard
| Challenge: | a weakly-supervised approach is needed to verify factual consistency . auxiliary span extraction tasks are useful for verifying factual consistent summaries . |
| Approach: | They propose a weakly-supervised approach for verifying factual consistency . they transfer the model to summaries generated by several neural models . |
| Outcome: | The proposed approach outperforms models trained with strong supervision on source documents and human evaluations. |
Automated Fact-Checking of Claims from Wikipedia (2020.lrec-1)
Copied to clipboard
| Challenge: | Fact checking datasets such as FEVER and SNLI suffer from limited applicability due to synthetic nature of claims and/or evidence written by annotators that differ from real claims and evidence on the internet. |
| Approach: | They present a dataset of 124k+ triples consisting of a claim, context and an evidence document extracted from English Wikipedia articles and citations. |
| Outcome: | The proposed dataset is the largest fact checking dataset consisting of real claims and evidence to date. |
ZoFia: Zero-Shot Fake News Detection with Entity-Guided Retrieval and Multi-LLM Interaction (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) are limited by knowledge cutoff and can generate factual hallucinations when handling time-sensitive news. |
| Approach: | They propose a two-stage zero-shot fake news detection framework that uses a hierarchical salience and saliency-calibrated minimum margin of relevance algorithm to extract core entities accurately. |
| Outcome: | The proposed framework outperforms existing zero-shot baselines and even most few-shot methods on two public datasets. |
CUB: Benchmarking Context Utilisation Techniques for Language Models (2026.acl-long)
Copied to clipboard
Lovisa Hagström, Youna Kim, Haeun Yu, Sang-goo Lee, Richard Johansson, Hyunsoo Cho, Isabelle Augenstein
| Challenge: | Existing language models (LMs) can be distracted by irrelevant contexts or ignore relevant information that contradicts outdated parametric memory. |
| Approach: | They develop a benchmark to help diagnose CMTs under diverse noisy context conditions within retrieval-augmented generation (RAG) they find that most existing CMT struggle to handle the full spectrum of context types encountered in real-world RAG scenarios. |
| Outcome: | The proposed benchmark compares seven state-of-the-art methods across three datasets and tasks, and shows that many lack the robustness needed to handle the full spectrum of context types encountered in real-world RAG scenarios. |